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Mental health is an important aspect in realizing overall health. For this 
reason, non-invasive medical equipment is needed for people with mental 
disorders. This study aimed to create a psychological health diagnostic tool 
by detecting auras using a thermal camera from facial objects. The 
contribution of this study is that the tool can detect the patient's aura without 
physical contact so that the patient is more comfortable and does not feel 
invaded. This research designed a system for detecting electromagnetic wave 
radiation energy emitted by the body using a thermal camera. Face detection 
in the input image was performed using a convolutional neural network 
(CNN) model single shot multibox detector (SSD), which is one of the CNN 
models that implements a bounding box to estimate the localization of 
detected objects. In this case, system testing was used to evaluate the 
performance of the CNN system algorithm for aura detection in terms of 
color (main color or average color). The results obtained were detectibility 


by 80%, selectivity by 88.88%, precision by 70%, sensitivity by 87.5%, and 
accuracy by 63.63%. The design of the aura detection system in this study 
will make it easier for psychiatrists and psychologists to help make a non- 
invasive diagnosis. 
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1. INTRODUCTION 

Mental health, in the forms of mental disorder, currently becomes a concern for the Indonesian 
government. According to the data issued by the ministry of health’s basic health research (Riskesdas 
Kemenkes), 282,654 household members, or 0.67 percent of Indonesian people experienced mental disorders 
in 2018. Mental health is an important aspect of realizing overall health. In addition, it is also important to 
pay attention to physical health, fear of a disease such as a cough, migraine, fever, or bullying, that can also 
be one of the factors causing mental disorders [1]-[4]. About 450 million people are suffering from mental 
and behavioral disorders worldwide [5], [6]. According to WHO Asia Pacific Region (WHO SEARO), the 
highest number of depressive disorder cases is in India (56,675,969 cases or 4.5% of the total population), 
while the lowest cases is in Maldives (12,739 cases or 3.7% of the population). In order to provide mental 
health services, medical equipment that can diagnose people with mental disorders effectively and non- 
invasively is needed. In this case, diagnostic equipment for mental disorders is still relatively rare and very 
expensive, in addition to the periodic fees that must be paid for every purchase of software application 
extensions. Therefore, it is very important to provide diagnostic equipment software for mental disorders at a 
low price and more effective. Several previous articles have discussed aura detection based on biometric 
traits such as fingerprint, voice, speech, face, and retina [7]-[9]. 
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In a previous study carried out by Fabijaeska and Sankowski [10], the aura removal algorithm 
method for aura phenomena was carried out for images at high temperatures. In the research, the specimen 
was heated to a high temperature until the light was emitted in the spectrum, causing the illumination of the 
photosensitive element of the charge coupled device (CCD) camera to saturate. Living organisms have a 
special field called the biofield that has a different structure in each layer of color, sound, shape, and structure 
called Aura [11]. Furthermore, Benke ef al. [12] studied the characteristics of aura phenomena with 60 
patients with heart disease and 40 patients with vasogespal syncope. The method used is a detailed 
anamnestic exploration. In addition, image processing has also been used by other previous researchers [13]-[15]. 
For example, Lee et al. [16] employed gas discharge visualization (GDV) for imaging techniques that are 
captured by biophotons and then emitted by the human body; this method is quite good for the quality of the 
reconstructed image [17]. Another research conducted by Poojay and Srinivas [18] reviewed various 
researchers about image analysis based on aura images. They stated that humans consist of many mysteries 
so that research on aura images can understand the individual’s state of mind and health conditions. 
Furthermore, Rajesh eft al. [19] through their study used aura images for medical diagnosis. In this case, 
experiments were carried out with 40 normal and abnormal subjects through a method where aura images 
were captured by mapping the energy of the five hand meridian points [20]. In addition, Rajesh [14] utilized 
the support vector machine (SVM) method to study patterns to recognize image textures based on the gray 
level aura matrix (GLAM). The success of this system resulted in an average value of 92%. However, this 
system is limited by distance, in which when the measurement distance is close, the texture will be more 
similar and vice versa. 

Based on several studies on aura detection that have been described above, no one has ever used a 
thermal camera to capture aura images and display thermal images using the convolutional neural network- 
single shot multibox detector (CNN-SSD) method. Furthermore, through the combination of the concepts 
proposed by Stefan Bolzman and Plank, it is possible to analyze the field energy and the wavelength of the 
aura in the form of electromagnetic waves emitted from a person’s body. Several researchers from various 
countries have developed image analysis based on image aura [21]-[24]. In this case, Poojay and Srinivas [18] has 
applied it using various patient objects. However, to the best of the author's knowledge, several authors have not 
investigated the aura image to detect mental disorders [18]. Therefore, the purpose of this study was to create 
a diagnostic tool for people with mental disorders by detecting auras using light spectrum analysis. The 
method used in this study was a camera to capture the object's posture and taps the aura object through the 
face. After that, the aura measurements were analyzed by light spectrum analysis. The output of the camera 
in the form of photos will open the interface which will then be displayed to the PC. This study can 
determine the presence of aura emitted from the human body, especially for those who experience mental 
health. The contributions of this research are: 

— The tool can detect the patient’s aura without physical contact so that the patient is more comfortable. 

— The tool can provide comfort for patients because the examination was carried out in a non-invasive 
manner. 

— The tool can help medical personnel to analyze mental disorders more easily using modern technology. 


2. RESEARCH METHOD 
2.1. Data processing 

This research created a system that can detect electromagnetic wave radiation energy emitted by the 
body with a thermal camera. The initial stage was to detect facial temperature in humans. The input used by 
the system was in the form of a thermal image obtained from taking pictures using a thermal camera. 
Convolutional neural network is a method that belongs to the feed forward neural network class [25] which 
was inspired by the visual cortex of the brain and specifically for processing data that has a grid structure as 
described in Figure 1. 

Face detection in the input image was performed using the CNN-SSD. In this case, SSD is one of 
the CNN models that implements bounding boxes to estimate the localization of detected objects. It has 
several advantages including speed and accuracy in recognizing objects compared to other models. Figure | 
shows that the face temperaturewas used as input, then the image preprocessing was carried out using a data 
set. Furthermore, the feature extraction process was carried out, continued by classification based on the 
CNN using the 2 convolution layer model to determine face detection, which can then be calculated radiation 
energy (Auras). CNN has several types of layers that were used. The convolution operation imposed on the 
function s(t) with weights (or often called kernels) w(t), x*w, is defined mathematically in (1). 


s(t) = (x*w) (¢) () 
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The s(t) function gives a single output in the form of a feature map. The first argument is input 
which is x and the second argument is w as kernel or filter. If you see the input as a two-dimensional image, 
you can say t is a pixel and replace it with i and j. Therefore, the operation for convolution to inputs with 
more than one dimension can be written as (2). 


SG@jf) = K*tD Gj) =2msn (i+mj+n) *K (mn) (2) 


based on the (1) and (2) above, the basic calculation is in the convolution operation, where i and j are the 
pixel of the image. Meanwhile, m and n are the dimensions of a kernel. These calculations are cumulative 
and appear when K is the kernel, then I is the input. As an alternative, the convolution operation can be called 
a matrix multiplication between the input image and the kernel where the output was calculated by dot 
product. After the face was detected, the maximum facial temperature data would be extracted. After that, the 
system displayed the facial temperature that was successfully extracted. Then, by programming stefan 
boltman’s concept/formula, the aura detection process was carried out. The workflow of the system can be 
seen in Figure 2. In this design, a thermal camera was utilized to see the energy wave radiation emitted by the 
body (aura), as supported by stefan boltman’s concept, namely (3). 


P=eo0A (3) 


P=the average radiant energy emitted; e=emissivity; o=Stefan Boltman’s constant=5.67 x 10° (W/m?K*); 
A=the surface area of the object; T=the temperature of the object (°C) and the max plank formula, namely (4). 


E =1,24.10-6/Aa (4) 


E=radiant energy; A=wave 
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Figure 1. The design of aura detection based on the convolution neural network 


2.2. Pre-processing data 

The thermal image used had a resolution of 220x90 (10,800 pixels). The thermal image was taken 
using a UTi thermal camera circling the subject’s face. Images provided by this data set have not been 
annotated. The dataset was further divided into training set and test set. The training set consisted of 400 
images, while the test set consisted of 80 images. The illustration of preprocessing can be seen in Figure 2. 

Based on the explanation on Figure 2, it is necessary to annotate or label the dataset before it was 
used as input, so that the machine can easily understand the pattern of the image. Furthermore, the training 
data augmentation is a deep learning process, which requires more data compared to the other types of 
machine learning. Through data augmentation, new data can be artificially created from the existing training 
data. Horizontal flip, image scaling, and image rotation were implemented in the training set. After the 
augmentation, the training set was 1,780 images, while the testing set was 108 images. In this case, tensor 
flow requires a label file to map each label used to an integer value. This map label was used in the model 
training and detection processes. Creation of tensorflow record files, the annotated image that has been 
created is converted to the tensorflow records (TFRecord) file format. TF record is a file format for storing 
data provided by tensorflow. Creation of region of interest (ROI) detection model was included in this study, 
where the detection was used on the face so that the temperature detection process could be carried out more 
quickly and easily without direct contact. The facial ROI detection model was created by training a pre- 
trained SSD-model. The model was retrained to be able to detect faces in thermal images using the 
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tensorflow object detection API. Furthermore, the protobuf file was also used to configure the training and 
evaluation processes. 


Testing 


Training 
Data set Data set 
annotations annotations 


i — 


Tensor flow record 


a 


Face detection 


Data set 


augmentation 


Figure 2. Preprocessing data of aura detection 


3. RESULTS AND DISCUSSION 

This study aimed to create a diagnostic tool for people with mental disorders by detecting auras 
using light spectrum analysis. The results of the research that have been achieved were the results of module 
design in the form of a human body aura tool module in the form of electromagnetic waves. Furthermore, the 
tool has been tried on during the data measurement to see the performance of the sensor used. 


3.1. Acquisition of prepossessing data 

The results of measuring the temperature of each person showed a different range. Therefore, the 
reference data that can be processed are also different from one data to another. Hence, before inputting the 
image data into data preprocessing, data obtained from a thermal camera image with a pixel size of 220x90 
was first carried out. Furthermore, the contours/points of the image were determined to determine RGB so 
that temperatures can be obtained by determining the maximum and minimum temperatures. In this case, a 
certain temperature was obtained based on the contour we wanted. In collecting this data, the image contour 
was on the forehead/head contour. Figure 3 shows the process of taking image contours for preprocessing 
data. In Figure 4, image data collection on one of the respondents and the determination of the image contour 
point obtained RGB of 232,56,88 at a temperature of 34.45 °C with an average color of intense red. 
Furthermore, the same procedure can be done with the other images. In this research, contour data collection 
was carried out on | image for 11 respondents. 

Figure 4 is an aura report from one of the respondents (RO1) taken from the aura camera as a gold 
standard. There is one main color or the average color of the aura, which is intense red. This color reflects the 
main characteristics of a person’s personality or mental health. The images from aura camera were used as 
the gold standard for testing system performance. 
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Figure 3. Taking the image of respondent ROI from the thermal camera 
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Figure 4. Taking the image of respondent RO1 from aura camera-gold standard 


3.2. Performance test of thermal camera 

Retrieval of aura measurement data in the form of electromagnetic wave energy emitted from the 
human body, was carried out directly on 11 respondents to test the performance of electromagnetic wave 
sensors in detecting. The following is how to measure the energy field 6 times for each of 11 respondents 
aged between 30 and 70 years using the aura detection module that has been made described in Figure 3. To 
obtain the amount of energy emitted from the body (aura), data processing was carried out using the 
application of the concept of stefan boltman’s law as in (1) above and plank’s law as in (2). Based on the data 
processing, the following results were obtained. Table 1 shows the result of the measurement of 
electromagnetic waves energy emitted from the body (Aura) using the concept proposed by stefan boltman’s 
law with temperature parameters using a thermal camera. In this case, through the combination of Plank’s 
law concept, the wavelength can be determined. 
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Table 1. Measurement results of temperature, energy field and wavelength 


No. Respondent Temp °C Energy (eV) Wavelength (A) 


ROI 
RO2 
RO3 
RO4 
ROS 
RO6 
RO7 
RO8 
ROO 
R10 
R11 


os 
BoC OIAMNMEYWN > 


34.75 
31.63 
32.73 
33.32 
32.82 
32.32 
32.34 
32.59 
32.75 
33.79 
33.42 


473 
4.54 
4.61 
4.64 
4.61 
4.58 
4.58 
4.60 
4.61 
4.67 
4.67 


2.62E07 
2.73E-07 
2.69E-07 
2.67E-07 
2.69E-07 
2.71E-07 
2.71E-07 
2.70E-07 
2.69E-07 
2.65E-07 
2.67E-07 


In order to more clearly describe the relationship between temperature parameters and those emitted 
from the body, the following Figure 5 shows the graph of temperature and energy. Based on Figure 6, the 
energy emitted by the body is directly proportional to the temperature emitted by the body. body. The 
following is a graphical image of the wavelengths of the 11 respondents as described in Figure 5. 
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Figure 5. Graph of temperature and energy measurement 
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Figure 6. Graph of the wavelengths emitted from a body 


3.3. Testing for performance system 


System testing was carried out to evaluate the performance of CNN system algorithms for aura 
detection in terms of color (main color or average color). This was done by calculating the confusion matrix 
which can represent predictions and actual conditions from gold standard data. In measuring performance on 
the confusion matrix, there were several performance metrics that are commonly and frequently used, those 
are detectability, selectivity and accuracy, precision, as well as sensitivity. Furthermore, Table 2 presents the 
result of the confusion matrix to determine system performance in image processing from a thermal camera. 
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Table 2. Performance system in aura color detection from thermal camera images 


No. Respondent Thermal camera System Result 
1. ROI Intense red Intense red TP 
2. RO2 Orange Green yellow FN 
3: RO3 Golden orange Golden orange TP 
4. RO4 Orange Orange TP 
5 ROS Golden Golden TP 
6 RO6 Yellow Green FP 
7 RO7 Red yellow Green yellow FP 
8 RO8 Green yellow Green yellow TP 
9 ROO Pink Pink TP 
10 R10 Golden orange Golden orange TP 
11 R11 Intense red Intense red TP 


On the matrices above, we obtained detectability=TP/(TP+FP)=8/(8+2)=0.80=80% and 
selectivity=TP/(TP+FN)=8/(8+1)=0.88=88.88%. Furthermore, the aura camera is the gold standard and was 
employed to see the performance of the testing system, as shown in Table 3. 


Table 3. Aura color detection system performance with aura camera as gold standard 


No. Respondent _ System Gold standart Result 
1. ROL Intense red Intense red TP 
2. RO2 Green yellow Blue FP 
3. RO3 Golden orange Golden orange TP 
4. RO4 Orange Red FP 
5. ROS Golden Golden TP 
6. RO6 Green Aqua marine =‘ FP 
7 RO7 Green yellow Green yellow TP 
8 R08 Green yellow Green yellow TP 
9 ROO Pink Orange FN 
10 R1O Golgen orange Golden orange TP 
11 R11 Intense red Intense red TP 


Based on Table 3, we obtained precision=(TP)/(TP+FP)=(7)/(7+3)=0.70=70%; 
sensitivity=(TP)/TP+FN)=(7)/(7+1)=0.875=87.5%; and = accuracy=(TP+TN)/(TP+FP+FN+TN)=(7+0)/ 
(7+3+1+0)=0.636=63.63%. Furthermore, the testing system performance was determined by using the aura 
camera as the gold standard. According to the results obtained, it is considered the CNN-SSD method for 
aura detection with thermal camera images in this study needs to be improved because the resolution was not 
high enough. Aura detection using thermal aura has never been done by researchers before. The design of the 
aura detection system in this study will make it easier for psychiatrists and psychologists to help make a 
diagnosis. 


4. CONCLUSION 

The aim of this research is to create a mental/psychological health diagnosis tool by detecting auras 
using a thermal camera from facial objects. After completing the data preprocessing, training and system 
testing, it can be concluded that the CNN-SSD method can be implemented properly to detect auras in 
accordance with the purpose of this study, which is to design a system to detect aura emitted by the body 
through image analysis of thermal camera images. The results obtained from this study are that aura, which 
can be described as energy of light waves emitted from the body, can be detected with a thermal camera by 
analyzing the spectrum of light and temperature. The energy emitted by the body is directly proportional to 
the temperature emitted by the body. Furthermore, the results of the wavelength of light emitted by 11 
respondents were 2.56.10-7 m-2.74.10-7 m. Based on the results of the wavelength range; the aura emitted by 
the body is an electromagnetic wave (infra red) that can be detected using thermal images. System testing is 
used to evaluate the performance of CNN system algorithms for aura detection in terms of color (main color 
or average color). This is done by calculating the confusion matrix which can represent predictions and actual 
conditions from gold standard data. The results obtained are detectability by 80%; selectivity by 88.88%; 
precision by 70%; sensitivity by 87.5%; and accuraracy by 63.63%. This system can be developed in further 
research by improving the accuracy. This can be done by increasing the resolution of the camera used and the 
number of input images and test data. Furthermore, in order to improve the design of this aura detection and 
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perfect it into a diagnostic tool, it is necessary to combine electrophotonic sensors and cameras as well as a 
data base for psychological decisions. 
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